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DETAILED ACTION 
Drawings 

1. The drawings are objected to under 37 CFR 1. 83(a). The drawings must 
show every feature of the invention as specified in the claims. Therefore, the 
underexposed image (II) of Figure 1 should be represented by some image with 
a visible contrast. A clear resolution must be present otherwise; Figure 1 would 
be interpreted as a blank diagram (image). Figures 2, 3, 9-1 1,12, 16, 18-28 
also have a very poor contrast and with reference to Figures 12, 16 and 18 
specifically, there is no visible image apparent. It is also difficult to understand 
applicant's invention without the appropriate distinguishing characteristics for 
these images. The aforementioned figures also must be shown or the feature(s) 
canceled from the claim(s). No new matter should be entered. 

Corrected drawing sheets in compliance with 37 CFR 1.121(d) are 
required in reply to the Office action to avoid abandonment of the application. 
Any amended replacement drawing sheet should include all of the figures 
appearing on the immediate prior version of the sheet, even if only one figure is 
being amended. The figure or figure number of an amended drawing should not 
be labeled as "amended.'' If a drawing figure is to be canceled, the appropriate 
figure must be removed from the replacement sheet, and where necessary, the 
remaining figures must be renumbered and appropriate changes made to the 
brief description of the several views of the drawings for consistency. Additional 
replacement sheets may be necessary to show the renumbering of the 
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remaining figures. Each drawing sheet submitted after the filing date of an 
application must be labeled in the top margin as either "Replacement Sheet" or 
^'New Sheet*" pursuant to 37 CFR 1.121(d). If the examiner does not accept the 
changes, the applicant will be notified and informed of any required corrective 
action in the next Office action. The objection to the drawings will not be held 
in abeyance. 

Claim Rejections - 35 USC §112 

2. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and 
distinctly claiming the subject matter which the applicant regards as his invention. 

3. Claims 9, 27 and 38 are rejected under 35 U.S.C. 1 12, second 
paragraph, as being indefinite for failing to particularly point out and distinctly 
claim the subject matter which applicant regards as the invention. 

As stated in claim 9 on page 37, line 9, in Claim 27 on page 41, line 9 
and at Claim 38 on page 44, line 14; applicant claims a method, computer 
program and apparatus which is further comprising modifying a color mapping 
function of the two images to cover a relatively larger range for a high contrast 
scene. How much larger is "relatively** larger? The fact is that claim language, 
including terms of degree may not be precise. The term "relatively^ is held as 
indefinite because there is a lack of a universal standard for measuring the 
degree intended for relatively larger to one of ordinary skill in the art. 
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Claim Rejections - 35 USC §102 

4. The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in 
this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a 
printed publication in this or a foreign country, before the invention thereof by the applicant for a 
patent. 

(b) the invention was patented or described in a printed publication in this or a foreign countiy or 
in public use or on sale in this countiy, more than one year prior to the date of application for 
patent in the United States. 

(e) the invention was described in (1) an application for patent, published under section 
122(b), by another filed in the United States before the invention by the applicant for patent 
or (2) a patent granted on an application for patent by another filed in the United States 
before the invention by the applicant for patent, except that an international application 
filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application 
designated the United States and was published under Article 21(2) of such treaty in the 
English language. 

5. Claims 1, 5-8, 10, 12, 15, 16, 26, 28, 30,37, 39 and 40 are rejected 
under 35 U.S.C. 102(e) as being anticipated by Chen (US 6,556,704). 
Regarding Claim 1: Chen discloses providing two images of a same scene 
(Refer to Figure 1, numeral 13a and 13b); determining a spatial coherence and 
color statistics of the two images ("...(a) generating intensity parameters 
corresponding to image features in each of two intensity images of a scene...'* at 
column 3, line 33); and utilizing the determined color statistics and spatial 
coherence to enhance one of the two images (''the intensity parameters in one 
image pairing with intensity parameters in the other image to form pairs of 
intensity parameters indicative of potential correspondence between features in 
the two intensity images...'' at column 3, line 34). 
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Regarding Claim 26: Chen discloses one or more computer-readable media 
having instructions stored thereon that, when executed, direct a machine to 
perform acts comprising: providing two images of a same scene; determining a 
spatial coherence and color statistics of the two images; and utilizing the 
determined color statistics and spatial coherence to enhance one of the two 
images (""A compact disk-read only memory (CD-ROM) 122 is connected to the 
microprocessor based unit 1 12 for receiving software programs and for 
providing a means of inputting the software programs and other information to 
the microprocessor based unit 112 via a compact disk 124, which typically 
includes a software program." at column 6, line 6). 

Regarding Claim 37: Chen discloses an apparatus comprising: means for 
providing two images of a same scene (Refer to Figure 1, numeral 12; two 
. vertically mounted cameras 11a and lib); means for determining a spatial 
coherence and color statistics of the two images ("The computer system 110 
includes a microprocessor-based unit 1 12 for receiving and processing software 
programs and for performing other processing functions.*" at column 5, line 61); 
and means for utilizing the determined color statistics and spatial coherence to 
enhance one of the two images ("A display 1 14 is electrically connected to the 
microprocessor-based unit 1 12 for displajdng user-related information 
associated with the software. A keyboard 1 16 is also connected to the 
microprocessor based unit 1 12 for permitting a user to input information to the 
software.'' at column 5, line 64). 



Application/Control Number: 10/817,633 Page 6 

Art Unit: 2624 

Regarding Claim 6: Chen discloses wherein the two images are related by the 
color statistics and the spatial coherence (*...(a) generating intensity 
parameters corresponding to image features in each of two intensity images of 
a scene..." at column 3, line 32), 

Regarding Claims 5 and 7: .Chen discloses wherein an underexposed one of 
the two images is enhanced and downloading the two images to a general- 
purpose computer, the general-purpose enhancing one of the two images ("In a 
typical implementation of the invention, the computer program product bearing 
the inventive algorithms would either be provided directly to a user... or it 
would be used in a shared setting, where a customer would bring pictures 
and/ or negatives ...for scanning and enhancement, or would directly enter 
digital scan data into the shared computer. Alternatively, the algorithms could 
be made available in a web-based version of the product, where either the 
algorithms are downloaded via the network connection to the user or the 
algorithm computation is done on a server in a web-based environment." at 
column 12, line 18). 

Regarding Claim 8: Chen discloses equally resembling claimed elements as 
mentioned in claims 6 and 7, therefore claim 8 is rejected for the same 
reasoning's as stated above in claims 6 and 7. 

Regarding Claims 10, 28 and 39: Chen discloses utilizing color histogram 
equalization to determine the color statistics computing a histogram of the 
number of the feature points that still have depth values in a column-wise 
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fashion, therefore to further deprive the depth values at feature points that do 
not belong to the majority of the remaining feature points that have depth 
values." at column 4, line 8). 

Regarding Claim 12, 30 and 40: Chen discloses utilizing spatial region 
matching to determine the spatial coherence ("...(b) eliminating one or more 
pairs of intensity parameters based on one or more constraints related to the 
feasibility of a valid match between the pairs of intensity parameters...'' at 
column 3, line 38). 

Regarding Claim 15: Chen discloses, wherein the method is performed by a 
digital camera ("It is also instructive to note that the images are either directly 
input into the computer system (for example by a digital camera) or digitized 
before input into the computer system (for example by scanning an original, 
such as a silver halide film)." at column 5, line 51). 

Regarding Claim 16: Chen discloses one or more computer readable media 
storing computer executable instructions that, when executed, perform the 
method as recited in claim 1, wherein the one or more computer readable 
media are incorporated inside a digital camera ("Any images stored in the PC 
card 130, the floppy disk 126 or the computer disk 134, or input through the 
network connection 127, may have been obtained from a variety of sources, 
such as a digital camera (not shown) or a scanner (not shown)." at column 6, 
line 32). 
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Claim Rejections - 35 USC §103 

6. The following is a quotation of 35 U.S.C, 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtsuned though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived by 
the manner in which the invention was made. 

7. Claims 2-4, 9, 17-20, 22, 24, 25, 27, 32, 33, 35, 38 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Chen (US 6,556,704) in combination 
with Hamilton (US 6,075,889). 

Regarding Claim 2: 

Chen discloses providing two images of a same scene; determining a 
spatial coherence and color statistics of the two images; and utilizing the 
determined color statistics and spatial coherence to enhance one of the two 
images. 

Chen does not expressly disclose that one of the images [is] taken in dim 
lighting conditionfs], 

Hamilton teaches wherein the images are taken in a dim lighting 
condition ("The exposure section 10 exposes the image sensor 12 to image light 
for a time period dependent upon exposure requirements, for example, a time 
period between 1/1000 second and several seconds." at column 4, line 4). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to utilize taking one image with dim lighting 
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condition[s] as taught by Hamilton and adding that condition to the one of two 
images of the same scene as disclosed by Chen because the light exposure time 
period will allow the image sensor as taught by Hamilton to gage the 
appropriate color statistics and spatial coherence and produce the most 
optimal luminance correction possible. 
Regarding Claim 3: 

Chen discloses providing two images of a same scene; determining a 
spatial coherence and color statistics of the two images; and utilizing the 
determined color statistics and spatial coherence to enhance one of the two 
images. 

Chen also does not expressly disclose that one of the images [is] is 
underexposed. 

Hamilton teaches wherein one of the two images is underexposed 
Although not shown, the exposure section 10 includes conventional optics for 
directing the image light through a diaphragm, which regulates the optical 
aperture, and a shutter, which regulates exposure time.'' at column 3, line 6). 
Regarding Claim 4: Hamilton teaches wherein one of the two images is blurred 
{''A digital image from the image buffer 18 of FIG. 1 is applied to a blur block 
42..." at column 5, line 2). 
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Regarding Claims 9, 27 and 38: 

Chen discloses means for providing two images of a same scene; means 
for determining a spatial coherence and color statistics of the two images; and 
means for utilizing the determined color statistics and spatial coherence to 
enhance one of the two images. 

Chen does not expressly disclose means for modifying a color mapping 
function of the two images to cover a relatively larger range for a high contrast 
scene. 

Hamilton teaches means for modifying a color mapping function of the 
two images to cover a relatively larger range for a high contrast scene (Refer to 
Figures 3(a) and 3(b); finally an RGB values block 36 computes the image in 
Red(R), Green (G), Blue (B) format which are used for an image display or for 
making a hard copy output... The purpose ... is to remove the high frequency 
components from the digital image.*" at column 4, line 57). 
It would have been obvious to one of ordinary skill in the art, at the time the 
invention was made to modify the color mapping function as taught by 
Hamilton to conform with the design of utilizing the color statistics and spatial 
coherence to enhance one of the two images as disclosed by Chen because the 
manipulation of the color spaces allows the user to remove artifacts and also 
creates an environment where the apparatus can process user information in a 
variety of ways. 



Application/Control Number: 1 0/81 7,633 Page 1 1 

Art Unit: 2624 

Regarding Claims 17 and 32: 

Chen discloses providing two images of a same scene; determining a 
spatial coherence and color statistics of the two images; and utilizing the 
determined color statistics and spatial coherence to enhance one of the two 
images. 

Chen does not expressly disclose providing an underexposed image of a 
scene and providing a blurred image of the same scene. 

Hamilton teaches providing an underexposed image of a scene ("Although 
not shown, the exposure section 10 includes conventional optics for directing 
the image light through a diaphragm, which regulates the optical aperture, and 
a shutter, which regulates exposure time."" at column 3, line 6); providing a 
blurred image of the same scene (A digital image from the image buffer 18 of 
FIG. 1 is applied to a blur block 42 which computes low frequency luminance 
values. Reference will now be made to FIG. 4 where a 5x5 blur kernel 
arrangement is shown." at column 5, line 2). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to add an underexposed image and a blurred image of 
a scene as taught by Hamilton as the two scenes of interest as disclosed by 
Chen because these multiple forms of images will resolve the photographic 
effects of most cameras and their similar pitfalls. 
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Regarding Claim 32: 

Hamilton teaches one or more computer-readable media having instructions 
stored thereon that, when executed, direct a machine to perform acts 
comprising: providing an underexposed image of a scene; providing a blurred 
image of the same scene; determining a spatial coherence and color statistics of 
the images; and utilizing the determined color statistics and spatial coherence 
to enhance the underexposed image ("The present invention can be embodied 
in a computer program stored on a computer readable product such as, for 
example, magnetic storage media, such as a magnetic disk (for example, a 
floppy disk), magnetic tape, optical disks, optical tape, or machine readable 
memory,'' at column 9, line 36). 

Regarding Claim 18: Hamilton teaches wherein the images are taken in a dim 
lighting condition wherein the images are taken in a dim lighting condition 
C^The exposure section 10 exposes the image sensor 12 to image light for a time 
period dependent upon exposure requirements, for example, a time period 
between 1/1000 second and several seconds,'* at column 4, line 4). 
Regarding Claim 19: Chen discloses wherein the images are taken 
successively in a short interval (" In practice, the cameras are arranged in such 
a way that the value of one of the disparities is always considered to be zero.*" 
at column 6, line 64). 

Regarding Claims 20 and 33: Chen discloses utilizing color histogram 
equalization to determine the color statistics C"... computing a histogram of the 
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number of the feature points that still have depth values in a column-wise 
fashion, therefore to further deprive the depth values at feature points that do 
not belong to the majority of the remaining feature points that have depth 
values.*^ at column 4, line 8). 

Regarding Claims 22 and 35: Chen discloses utilizing spatial region 
matching to determine the spatial coherence ("...(b) eliminating one or more 
pairs of intensity parameters based on one or more constraints related to the 
feasibility of a valid match between the pairs of intensity parameters...'' at 
column 3, line 38). 

Regarding Claim 24: Hamilton discloses, one or more computer readable 
media storing computer executable instructions that, when executed, perform 
the method as recited in claim 17 ("The present invention can be embodied in a 
computer program stored on a computer readable product such as, for 
example, magnetic storage media, such as a magnetic disk (for example, a 
floppy disk), magnetic tape, optical disks, optical tape, or machine readable 
memory."" at column 9, line 36). 

Regarding Claim 25: Chen discloses, wherein the method is performed by a 
digital camera (''It is also instructive to note that the images are either directly 
input into the computer system (for example by a digital camera) or digitized 
before input into the computer system (for example by scanning an original, 
such as a silver halide film).'' at column 5, line 51). 
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8. Claims 1 1, 14, 21, 29, 34 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Chen (US 6,556,704) in combination with Hamilton (US 
6,075,889) and Kang (US 6,879,731), 
Regarding Claim 14: 

Chen discloses providing two images of a same scene; determining a 
spatial coherence and color statistics of the two images; and utilizing the 
determined color statistics and spatial coherence to enhance one of the two 
images. 

Chen does not expressly disclose using an exposure bracketing feature of 
a digital camera to provide the two images, 

Kang in the same field of luminance correction teaches using an 
exposure bracketing feature of a digital camera to provide the two images 
(Many still picture cameras today allow for auto bracketing. When auto 
bracketing a scene, the camera determines the correct exposure via the current 
metering mode, and then additionally captures the scene at shorter and longer 
exposures. The present HDR video system modifies this approach in that 
instead of bracketing with a fixed multiple of the middle exposure, a set of 
exposures more appropriate for the scene is automatically determined.'' At 
column 17, line 60). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to add an exposure bracketing feature as taught by 
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Kang to the apparatus and method of providing two images of the same scene 
as disclosed by Chen because at the time the invention was made, many 
picture cameras allow for automatic bracketing. This provides the user with an 
all inclusive apparatus and method of manipulating the data of the two images. 
Regarding Claims 11, 21, 29 and 34: Kang discloses utilizing color 
histogram equalization to determine the color statistics, wherein the color 
histogram equalization comprises: transferring the two images to a perception- 
based color space ("FIG. 4 is a combined intensity histogram of the two images 
shown in FIG. 3 in radiance space. The left hand side of the plot corresponds 
to the long exposure frame, while the right hand side of the plot corresponds to 
the short exposure frame." at column 12, line 45); clustering color distributions 
in the perception-based space (The HDR video generation system and process 
can also involve tone-mapping of the radiance map to convert it into an 8-bit 
representation of the HDR frame that is suitable for rendering and display.'' at 
column 10, line 66); performing histogram equalization in the perception-based 
space (Referring to FIGS. 5A-B, the first step in calculating exposure settings to 
be used in capturing subsequent frames is to compute an intensity histogram 
for each of a pair of immediately preceding, already captured frames (process 
action 500).'' at column 18, line 38); and transferring a result of the histogram 
equalization to a red-green-blue space ("The CIE space image is then converted 
to produce the final 8-bit range RGB image." at column 1 1, line 6). 
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Allowable Subject Matter 

9. Claims 13, 23, 31 and 36 are objected to as being dependent upon a 
rejected base claim, but would be allowable if rewritten in independent form 
including all of the limitations of the base claim and any intervening claims. 



Conclusion 

Any inquiry concerning this communication or earlier communications 
from the examiner should be directed to Mia M. Thomas whose telephone 
number is 571-270-1583. The examiner can normally be reached on Monday- 
Friday 8:30am-5pm. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Brian Werner can be reached on 571-272-7401. The 
fax phone number for the organization where this application or proceeding is 
assigned is 57 1-273-8300. 
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Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see 
http://pair-direct.uspto.gov. Should you have questions on access to the 
Private PAIR system, contact the Electronic Business Center (EBC) at 866-217- 
9197 (toll-free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 
9199 (IN USA OR CANADA) or 571-272-1000. 



Mia M Thomas 
Examiner 
Art Unit 2624 
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/Brian P. Werner / 

Supervisory Patent Examiner (SPE), Art Unit 2624 



